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Abstract 

In this paper, mathematical bases for non-adaptive joint source net- 
work coding of correlated messages in a Bayesian scenario are studied. 
Specifically, we introduce one-step Quantized Network Coding (QNC), 
which is a hybrid combination of network coding and packet forwarding 
for transmission. Motivated by the work on Bayesian compressed sensing, 
we derive theoretical guarantees on robust recovery in a one-step QNC 
scenario. Our mathematical derivations for Gaussian messages express 
the opportunity of distributed compression by using one-step QNC, as 
a simplified version of QNC scenario. Our simulation results show an 
improvement in terms of quality-delay performance over routing based 
packet forwarding. 



1 Introduction 

Unlike routing based [l] packet forwarding, network coding [2] offers advantages 
in terms of flexibility and robustness to deployment changes, for data gathering 
in sensor networks. Beyond that, in noisy networks, resistance to link failures 
and errors has made network coding a better alternative for packet forwarding, 
as a result of flow diversity in the network j3j. In this paper, we aim to exploit 
the possibility of non- adaptive distributed source coding by introducing a hy- 
brid network coding and packet forwarding transmission method for correlated 
sensed data (referred as messages in this paper). 

Compressed sensing concepts have been recently used in the networks to 
develop non-adaptive approaches for efficient communication [4j[5j. Distributed 
source coding [6] , embedded in network coding is recently studied in J7|8j , where 
the compressed sensing [9] is joined with network coding. Specifically, in [7], we 
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have proposed and formulated Quantized Network Coding (QNC), which incor- 
porates random linear network coding in real held and quantization to cope with 
the limited capacity of the links. By using restricted isometry property, theo- 
retical guarantees for robust ^-min recovery |7| in non-Bayesian QNC scenario 
was discussed in [10] . In Bayesian scenarios (where a priori is known beyond the 
sparsity domain) , to tackle the design of a practical and near optimal decoder, 
we proposed a Belief Propagation (BP) based Minimum Mean Square Error 



(MMSE) decoder 11 . In this paper, we provide theoretical bases to justify 
adopting Bayesian quantized network coding for correlated Gaussian messages. 
Specifically, we derive the required number of received packets for robust recov- 
ery of messages, implying an embedded distributed compression of messages, 
while keeping other advantages of network coding over packet forwarding. 

In section[2] we describe the assumptions on the random network deployment 
and the used model on near sparse messages. Then, in section [3] we introduce 
and formulate the one-step QNC, and derive theoretical guarantees for robust 
recovery of near sparse messages. Our simulation results are presented in sec- 
tion]^] which is followed by our conclusions in section [5] 



2 Problem Description and Notation 

Consider a directed network (graph), Q = (V, £), where V = {1, ••■ ,n}, and 
£ = {1, • • ■ , |£|}, are the sets of nodes and edges (links), respectively. Each edge, 
e, maintains a lossless communication from tail(e) to head(e), at a maximum 
rate of C e bits per use. This implies the same input and output contents for edge 
e G £, denoted by Y e (t), where t is the time index, representing transmission of 
a block of length L over the edges. The edges are uniformly distributed between 
the pairs of nodes. Explicitly, for each e G £ , we have: 

P(tail(e) = v,head(e) = v') = — r, Wv,v' G V, (1) 

V / n(n — 1) 

where » ^ The sets of incoming and outgoing edges of node v, are also 
defined as: 

In{v) = {e : head(e) = v}, (2) 

Out(v) = {e : tail(e) = v}. (3) 

Each node has a random information source (message) X v , where X v G K. 
These random messages, X_ = [X v : v G V], are correlated and their correlation 
can be modeled by using an orthonormal transform matrix, (f> nX m f° r which 
5 = (j> T X is almost fc-sparse: ~ k. In this paper, we assume S v 's are 

drawn from a two-state Gaussian mixture model, such that there is a random 
state Q v , for which: 

Q v = l -> S v ~N(0,a 2 s ), 

Q v = -> S v ~tf(0,o*), (4) 
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and a 2 and a 2 , with a 2 ^> a 2 , are the variances of large and small elements of 
near sparse domain. 

In the described noiseless network, we need to transmit messages, X v 's, to 
a single gateway (decoder) node, v € V and recover them with a small or no 
distortion. Performing distributed source coding [6j along packet forwarding is 
used to transmit correlated messages. But, dealing with the aforementioned net- 
work coding advantageous, we are interested to establish theoretical guarantees 
for joint source network coding, in the non-adaptive QNC scenario. 



3 One-Step Quantized Network Coding 

We have defined and formulated QNC and its Bayesian counterpart in [7 11 . 
In this section, we describe a special case of QNC, in which QNC is done only 
at the first time instance, i.e. t = 2. The resulting quantized network coded 
packets are then forwarded to the decoder node for recovery. In such scenario, 
we discuss robust recovery of messages and derive mathematical guarantees for 
local network coding coefficients. One-step QNC is a hybrid method which 
consists of two stages of transmission, as explained in the following. 

Assuming initial rest condition in the network, we have: Y e (l) — 0, Ve € £ . 
The messages, X v , are available for transmission at t = 1. Therefore, the passed 
packets between the nodes and received at t = 2 are only the raw quantized 
messages; i.e. Q e (X„)'s: 

y e (2) = Q e (Xtaa(e)), (5) 

where Q e (.) is the quantizer operator, corresponding to edge Denoting the 
quantization noise by N e (2), we have the following additive form: 

Y e (2)=X tail(e) +N e {2) 1 Vee£. (6) 

These quantized version of neighboring node messages are used to calculate a 
random linear combinations of them. Specifically, we define P v , as the one-step 
linear combination of messages, at node v, according to: 

Pv = Yl ^,e'Y e ,(2) + a v X v , Vv e V. (7) 

In |7]), the local network coding coefficients, j3 v _ e i and a v , are uniformly and 
randomly picked from {— k, k > 0. In order to prevent from over flow, we 
make sure that the normalization condition of Eq. 3 in [7 is satisfied. 

Some of these random linear combinations, P^'s, are then transmitted to 
the decoder node via packet forwarding scheme. Specifically, P v is sent to the 
decoder node with a probability of — , where m is the number of received packets 
at the decoder node, by time t. Representing the i'th received packet at the 
decoder node by Z_ tot (t), we have: 

{Z tot (t)}i = P v +N e (3), v^i, (8) 



lr The design of Q e (.) depends on the value of L and C e . 
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where v — > i means that P v is forwarded to decoder and corresponds to the i'ih 
received packet. Outgoing edge, e £ Out(v) is also the edge on which P v is sent 
out of node v, and therefore Q e (.) is used for quantization of P v . 

Because of the linearity of QNC scenario, we can reformulate Z_ tot (t) as: 



Z tot (t) = * tot (t)-X + N eSttot (t), 



(9) 



where ^tot(t) and jV efftot (i) are the total measurement matrix and the total 
effective noise vector, respectively, calculated according to: 



{*tot(t)k« 




V — t l, v — t V ■ 

v' —} i, v' = v, 
otherwise 



{iV cff , tot (t)} 4 = N e (3) + ^'(2), v ^ i. 



(10) 



(11) 



e' Eln(v) 



In (jlOh , v A v 1 denotes the existence of edge e' from v to v'. 

Motivated by compressed sensing theory [9], if we are able to recover the 
messages, X. from fewer number of received packets than the number of mes- 
sages, i.e. m < n, then we have been able to perform an embedded non-adaptive 
compression of correlated messages. 

Bayesian QNC was shown to be working better than packet forwarding by 
deducing the required number of received packets for robust recovery. In the 
following, we present a theorem which offers robust recovery guarantee for one- 
step QNC scenario, as a simplification of (full) Bayesian QNC. Specifically, 
it justifies the use of one-step QNC for transmission of near sparse Gaussian 
messages, in terms of the information content of received packets at the decoder; 
i.e. random linear combinations, iVs. 



Theorem 3.1 For the described one-step QNC scenario, with almost k-sparse 
messages, characterized as in section^ if 



2n 2 



(12) 



and, 



n ,' (l + 7) fc 3 n 2 log(fcn^) 
m = L>\ s — —, log(n) ) , 



(13) 



[i 2 n+\S\ 

where 7, fi > 0, one can decode X_, from the measurements of the form ^tnt(t)X , 
to X(t), such that: 

\X-X(t)\\ £ <^s, (14) 
with a probability exceeding 1 — n r . [^] 



2 A complete definition of 0(.) and o(.) can be found in Il2| 
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Proof 0TO prove the theorem, we begin by finding probabilistic bounds on the 
£2 and too norms of X_. Since <f> is an orthonormal matrix, ||2l||^ 2 — ||5|| f2 , 
implying the same probabilistic £2 norm of 2L, as that of 5. Therefore, by using 
Eqs. 8,10 in [13], we have: 

P(\\K\\l <ka 2 s ) = o{n^), (15) 

V{\\X\\l >2ka 2 s + (n-k)a 2 z ) = o{n^). (16) 

To obtain a bound on norm of X, we use Eq. 17 in |13| . However, since 
their discussion is for a canonical case of <f>, we need to take care of non-diagonal 
4>, in our case. Basically, the orthonormal <j) rotates the axes in S_ domain to X_ 
domain. Considering an extreme case in which all k non-zero ele men ts of £ are 



at their maximum possible limit (say yj2 \og(krCi)a s from (17) in 13 ), £2 norm 
of 5 is \Jk times that limit. Now, consider <fi rotates the axes such that we have 
an axis along the same direction as that extreme case of £[. In such case, the 
£00 norm of X corresponds to the £2 norm of S[ in the aforementioned extreme 
case, implying: 



P(||2CIL < V2felog(fcn> s ) > 1- V' (17) 

|| X ||2 

and < 21og(fcn 7 ), with overwhelming probability. 

IIAI U 2 

Assuming (without loss of generality) that P„'(2) is forwarded to decoder 
and corresponds to the i'th received packet, for {'I'tot (lira's, we have: 

P({*tot(*)}to=0) = P(v^v',Pe' :«4t/) 

= P(v^v') P(fie' :v^fv'\v^v') 



1 



iw (igi/")»(|gr/j 

n){ (\£\/n)\(^- n \) 



1-^(1-^). (18) 
nJ \ n(n — 1) / 

Since +k and — k are picked with the same probability for network coding 
coefficients, we have: 

P({*tot (*)}«« = +«) = P({*tot(*)}i« = -K) 

= ^-^P({*totW} te =0) 

1 i^.IU! 1*1 



2 2 V nJ\ n(n-l) 
n + \S\ 



K 



2n 2 

1 

~2 ' 



(19) 



'Our proof uses a similar approach as in the proof of theorem 1 in 13 . 
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For Vi, v, 1 < i < m, 1 < v < n, this implies: 



E[{*tot(*)}<«] = 0, 
E[{*totW}L] = 1, 
E[{* to t(t)}t] - « 2 



and therefore: 



k 2 • 2 ln(/cn 



4n 2 ln(fcn 7 ) 



(20) 



(21) 



n + \S\ 

By applying theorem 1 in 14] , using a similar reasoning as in the proof of 



theorem 1 in 13 , and choosing 



2k 



(22) 



we can finish the proof of our theorem. 



The following can also be established from theorem 3.1 



Theorem 3.2 For the described one-step QNC scenario, if (12) is satisfied, 
and, 

|£|=0(n 2 ), (23) 



TO = O 



^-^fc 3 log(fc^)log(n)). 



(24) 



where 7, pb> 0, then (14) holds with a probability, exceeding 1 — n 



Theorem 3.2 states that in a (relatively) highly connected network deploy- 



ment (high number of edges), we need smaller order of received packets, to, as 
the order of messages, n, to be able to recover the messages. This saving implies 
an embedded distributed compression, which is achieved without adapting the 
local network coding coefficients (We can pick a v 's and (3 v ,e's from { — 1, +1} and 
then scale the received packets by an appropriate constant result in the same 
^tot(i) as picking them from {— k, +k}). In other words, the total measurement 
matrix, resulting from one-step QNC scenario has similar characteristics as an 
appropriate measurement matrix for Bayesian compressed sensing. Therefore, 
a similar recovery guarantee can be offered for our case of one-step QNC. 



4 Simulation Results 

In this section, we compare the performance of one-step QNC with that of rout- 
ing based packet forwarding. To do so, different deployments of a sensor network 
with n = 100 nodes and \£\ = 400, 800 uniformly distributed edges are generated 
randomly. The edges are directed and can maintain a lossless interference-free 
communication of C e = 1 bit per use. Messages are also randomly generated 
using a random <f> according to the model of Eq. [4J for cr 2 = 1 and er 2 = 0.01, 
and different sparsity factors: - = 0.05,0.15. 
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For each deployment, one-step QNC is run by using a uniform quantizer of 
appropriate step size (depending on the value of block length, L and putting 
the dynamic range between — 4<r s and +4a s ). In a progressive manner, we use a 
routing based packet forwarding to deliver all P v 's to the decoder node. Then, at 
each time, t, the received packets up to t are used to recover the messages, which 
lets us obtain different quality and delay performance points. Specifically, we use 
both Belief Propagation (BP) based minimum mean square error decoding 
and £i-min decoding [7] to recover messages, from the received P„'s up to t 

In a different scenario, for each deployment, packet forwarding via optimal 
route is run and messages are delivered to the decoder node. Each message, X v: 
is quantized at its source by using a similar uniform quantizer as used in one- 
step QNC scenario. The delivered quantized messages up to time t are taken 
care for calculating the corresponding signal to noise ratio. Moreover, the routes 



If 



from nodes to the decoder node are calculated by using Dijkstra algorithm 15 



To obtain a quality-delay diagram of the performance, we find the best 
block length, L, for each SNR value, corresponding to each one-step QNC and 
packet forwarding scenario. The resulting delivery delays are then averaged 
over different deployments of network. The average SNR (quality measure) is 
depicted versus the average delivery delay (cost measure) , for 400 and 800 edges 
in Fig. I (a) | and Fig. |I (b)) respectively. 

As it was expected from the mathematical derivations of section [3j the aver- 
age delivery delay of one-step QNC for a given SNR is less than that of packet 
forwarding, in a wide range of SNR values. Smaller a sparsity factor, -, meaning 
higher level of correlation between messages, results in a better performance. 
Moreover, as shown in Figs. l(a)|l (b) larger number of edges (higher edge 



densities) increases the performance gap between the one-step QNC and packet 
forwarding scenarios. Although the resulting performance of one-step QNC may 



not look promising for all SNR values, as it was illustrated in 11 , using (full) 



QNC scenario helps us get a significant improvement for all SNR values. 



5 Conclusions 

Theoretical motivations behind Bayesian QNC are discussed by deriving math- 
ematical guarantees for robust recovery of near sparse Gaussian messages, in a 
simple QNC scenario, called one-step QNC. Our derived conditions for robust 
recovery, which bounds the 1^ norm of error, show that one-step QNC requires 
smaller order of received packets at the decoder than that of messages. This 
implies an embedded distributed compression, while there is no need to adapt 
the encoding with the correlation model of messages (i.e. sparsifying trans- 
form, </>). However, more mathematical works are still necessary to analyze the 
(full) Bayesian QNC scenario to obtain tighter bounds for robust recovery of 
messages. 
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Figure 1: Average SNR versus average delivery delay of one-step QNC and 
packet forwarding for different sparsity factors, — , and different number of edges. 
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